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Abstract 

Structures play a significant role in the field of signal processing. As a representative 
of structural data, low rank matrix along with its restricted isometry property (HIP) has 
been an important research topic in compressive signal processing. Subspace projection 
matrix is a kind of low rank matrix with additional structure, which allows for further 
reduction of its intrinsic dimension. This leaves room for improving its own RIP, which 
could work as the foundation of compressed subspace projection matrix recovery. In 
this work, we study the RIP of subspace projection matrix under random orthonormal 
compression. Considering the fact that subspace projection matrices of s dimensional 
subspaces in R w form an s(N — s) dimensional submanifold in M . NxN , our main concern 
is transformed to the stable embedding of such submanifold into Hl NxN . The result 
is that by 0(s(N — s) log N) number of random measurements the RIP of subspace 
projection matrix is guaranteed. 

Keywords: restricted isometry property, subspace projection matrix, low rank 
matrix, manifold stable embedding, compressive signal processing 


1 Introduction 

Signal structure has always been a key point in the field of signal processing. Structural 
data, such as sparse signal and low rank matrix, have been important research topics in 
compressive signal processing m- These structures invoke low intrinsic dimension, so the 
restricted isometry property (RIP) can be established to guarantee both exact and robust 
reconstructions from randomly compressed measurements mm- 

For a given s dimensional linear subspace S in an Euclidean space M. N , assuming that 
s is less than N, the subspace projection matrix is a low rank matrix with rather specific 
structure. In fact, it is not only symmetric, semi-definite, but also has merely eigenvalues 1 
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and 0. Such additional structure invokes lower intrinsic dimension than a general low rank 
matrix does, therefore theoretical improvement on the RIP can be expected. 

According to the basic ideas in compressive sensing [6], the RIP of subspace projection 
matrix could work as the foundation of compressed subspace projection matrix recovery. 
Considering the fact that subspace projection matrix has a one to one correspondence 
with subspace, the recovery from its compression could be viewed as compressed subspace 
estimation. Subspace estimation has been a concerning problem in signal processing and 
computer vision. In some scenarios, such as face recognition [7], motion segmentation [5], 
and visual tracking [9], the objects belong to subspaces with much lower dimension than 
the ambient space. In fact, subspace estimation from highly incomplete information has 
recently appeared as an attractive research topic mmm- 

For subspaces with a given dimension in M . N , their projection matrices form a manifold. 
There are significant and solid works in manifold-modeled signal recovery from randomly 
compressed measurements da na nans]. These works extend classic compressed sensing by 
generalizing low-dimension model from sparse signal to signal on low dimensional manifold, 
and study stable manifold embeddings and nonadaptive dimensionality reduction of data 
on manifold. One of the key ideas is to control the regularity of the manifold so that it 
is well-conditioned. The work in m utilizes an instructive quantity called the condition 
number of a manifold also known as the reach of a manifold, which studies submanifold 
extrinsically and unveils its Riemannian geometry properties [18]. 

In this work, we aim to study the RIP of subspace projection matrix under random 
orthonormal compression. A matrix manifold is used to model the set of subspace projection 
matrices. By investigating the differential structure and the condition number of such 
manifold, we are able to conclude that by 0(s(N — s)logN) random measurements the 
RIP of subspace projection matrices is guaranteed. 

2 Main result 

In this work, we study the RIP of subspace projection matrices under random orthonormal 
compression. 

Definition 1 The set of projection matrices corresponding to s dimensional subspaces in 
K n is defined as 

V N)S : = {P x = X{X T X)~ 1 X T : X e R Nxs , dim(span(X)) = s} 

= {P x = XX T : X € Gr/v )S }, (1) 

in which span(A) denotes the column space of X, and Grjv, s is the Grassmann manifold of 
s dimensional subspaces in . 
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Remark 1 Equation (jTJ) is obtained by ortho-normalizing the columns of X while keeping 
span(X) fixed. Because different choices of X do not change Px as long as span(X) is 
fixed, we have X G Grjv.s Ill- 

Remark 2 From Definition [IJ we know that, for a linear subspace S, its projection matrix 
is the matrix that a vectors G has to multiply when projected onto S. This is the reason 
that it is called a projection matrix. 


Vn,s is an s(N — s) dimensional submanifold in W NxN . The following theorem describes 
the RIP of Vx,s under random orthoprojector. 


Theorem 1 For fixed 0 < e < 1 and (3 > 0, assume that N > 3. Let A : M. NxN — > M m be 
a random orthoprojector with 

m -(^W3) 0 ( S( "- S)l0g f)' (2) 

If m < N 2 , then with probability exceeding 1 — e _ci ^ s ( iv_s ) ; in which c\ is a universal 
constant, the following property holds for every pair of Px, Py 6 Vn,s> Px A Py, 


(l- £ ) 


m < \\A(Px ~ Py)h < (i + £ ) ' Jm 


N \\Px — Py\\f 
Proof The proof is postponed to section [4} 


N 


(3) 


From Theorem [I] the number of measurements m > 0(s(N — s) log IV) is enough to 
guarantee the RIP under random orthonormal compression. For low rank matrices without 
further specific structure, the number of measurements should be no less than 0(sN log N) 
HJ. One may notice that when s <C N, the improvement from the latter to the former is not 
much. Although it is true in that case, the result in Theorem [l] does improve the scaling 
law of the number of measurements on s, and verifies the intuition that, compared with 
low rank matrices, subspace projection matrices have additional structure which is able to 
further reduce the number of compressed measurements needed for reconstruction. 

Heuristically, the scaling law m > 0((N — s)slogN ) is reasonable, in that the degree of 
freedom of a subspace projection matrix is s(N — s). It should be highlighted that although 
Theorem [l] is for an orthonormal random compression, the conclusion could be naturally 
extended to random compressions satisfying the concentration property [20J. 

In the following text, the establishment of Theorem [I] will be demonstrated. The first 
order differential structure and the condition number of Vn,s are studied in section |3j and 
then Theorem [l] is readily proved in section |4| 


3 Manifold of Subspace Projection Matrix 

Matrix manifold has been a powerful tool to structural matrix data recovery [21] . Typical 
matrix manifolds, such as the Grassmann manifold, the orthogonal group, and the Stiefel 
manifold, have been comprehensively studied, and one may read EafUES] for reference. 
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Figure 1: V 2.1 can be illustrated as a circle with radius l/y/2. 

In this section, we study the set of subspace projection matrices of s dimensional sub¬ 
spaces in K n denoted as Vn,s and defined in (jTj) . Because a subspace has a one to one 
correspondence with a projection matrix, and such correspondence is continuous, Vn,s is 
an s(N — s ) dimensional manifold which is homeomorphic to Gv]y,s- 

Preceding the calculation of the differential structure and the condition number, we may 
first illustrate Vn,s f° r specific N = 2 and s = 1. From the definition, we know that 

7\i = j[ xl,V2xix 2 ,xl] T : [xi,x 2 ] T 6 Gr 2> i| . 

Because Gr 2i i = S 1 , V 2.1 can be expressed as a circle with radius 1 /x/2 as shown in Fig. [I] 

3.1 Tangent space and normal space of Vn,s 

In this part, we study the first order differential structure of Vn,s■ The reason we need it is 
that the condition number, which is an important quantity used for the stable embedding 
of a manifold, will be defined by the normal bundle. 

Denote skew(IV) as the set of all IV x N skew symmetric matrices, and sym(IV) as the 
set of all N x N symmetric matrices. In the following lemma, the tangent space and the 
normal space at every point of Vn,s are unveiled. Remind that by stating Px = XX T , it 
is indicated that X £ Gin,s- 

Lemma 1 The tangent space of Vn,s a t a point Px = XX T is 

Tp x V n , s = {X ± KX t + XK t XJ : K £ M( Ar - s ) xs }, (4) 
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and the normal space is 


Np x Vn,s = skew(iV) LJ{S e sym(IV) : S = (P x - Px ± )S 0 , S 0 G sym(IV)}. (5) 
X± is the matrix such that [X, X_|_] T [X, A'jJ = Ifir- 


Proof Denote the sets in Q and ([5]) as U and V, respectively, both of which are subspaces. 

Suppose that Px(t) = is a curve on Vn,s with P X (o) = p x- The tangent 

vector along such curve at point P x is 


d. 
d t 


P x(t) 

t =o 


d X(t) 
d t 


X(0) T + X(0) 

t =o 


d X(t) T 
dt 


t =o 


+ xe, 


in which £ is a vector in 7xGr/v, s - Because these tangent vectors all belong to the tangent 
space of Vn,s at Px, we have 


Tp x V n , s DU = {£X t + ^ G T x G rjv , s }. 


To check that U1.V, notice that any symmetric matrix is orthogonal to any skew-symmetric 
matrix, and that 

tr ((X ± KX t + XK T Xl)(XX T - X ± Xl)S 0 ) 

= tr ((X ± KX t - XK T Xj)S 0 ) 

= tr(Xj _KX t S^) - tr(5 0 XP' T Xl) = 0. 


The dimension of V is 


N(N - l)/2 + s(s - 1) + (N - s)(N - s-l)/2 +N = N 2 -Ns + s 2 , 

and the dimension of U is Ns — s 2 , which is equal to the dimension of Tp x V x ,s- Therefore, 
we finish the proof that Tp x Vn,s = U and Mp x Vn,s = V. ■ 


3.2 Condition Number of Vn,s 

For the purpose of delineating the regularity of a manifold, a notion called the condition 
number of a manifold, also known as the reach of a manifold m, is introduced. 

Definition 2 JT7} i Let M be a compact Riemannian sub-manifold ofR N . The condition 
number is defined as 1 /r, where r is the largest number having property that the open normal 
bundle about A4 of radius r is embedded in M. N for all r < r. 

The condition number 1/r controls both local properties, such as the curvature of any 
unit-speed geodesic curve on the manifold, and global properties, such as how close the 
manifold may curve back upon itself at long geodesic distance [18| . However, from its 
definition, the condition number of a manifold is not easy to obtain in general. In this 
section, we focus on the condition number of the manifold Vn,s- I n the next section, we 
shall see that the following lemma is a key step to the main result. 
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Lemma 2 The condition number 1/r ofPx,s is V%- 

Proof The proof is based on the fact that r is the radius of the largest possible non-self- 
intersecting tube around the manifold. 

The distance between Px and the set of skew matrices is \\Px — 0\\f = \/s, in that 
\\P x \\f = sJtv(XX^XX^) = ^tr(X T X) = V~s. 

Thus, if the tube around the manifold intersects itself at a point of skew symmetric matrix, 
then the radius of the tube is no less than ^fs. 

For any X and Y in Gt N jS , denote X = ATY t - X ± Xj and Y = YY T - Y ± Y?. 
If the tube around the manifold intersects itself at a point of symmetric matrix $, then 
3 X, Y E Gr x,s such that 

XSi = YS 2 = $ E sym(IV). (6) 

Equation Q is equivalent to the condition that $ and X share the same eigenvector matrix 
X, and $ and Y share the same eigenvector matrix Y, which is not necessarily to be 
the same as X. One of the eigenspaces of X is span(X) corresponding to eigenvalue 1, 
and the other one is span(ATj_) corresponding to eigenvalue —1. Thus, X is composed of 
basis of span(X) and basis of span(XjJ. Without loss of generality, we can assume that 
X = [X, X±] and Y = [Y, Yj_]. Now we are able to prove that if ||$ — Px\\f < 1/V2 and 
||$ — Py\\f < l/\/2, then Q can not hold. 

First we consider the case where rank($) = s. Suppose that ([b]) holds, and ||$ — Px||f < 
1/V2 and ||$ — Py\\f <1/\/2. According to the discussion in the previous paragraph, we 
have that ||<3? — XA' T ||i? = ||Ai — A\\p and ||$ — Ty T || A ' = ||A2 — A\\p, in which A, Ai, and 
A2 are diagonal matrices. A = diag[1, • • • , 1,0, • ■ • , 0], in which the number of 1 is s. Both 
Ai and A2 have s non-zero elements and N — s zeros on the diagonal. Observe that if the 
non-zeros of Ai is the first s elements on its diagonal, then the non-zeros of A2 can not be 
the first s elements on its diagonal. Otherwise, X and Y span the same subspace. Thus, a 
contradiction comes from the fact that at least one of ||Ai — A||^? > 1 and ||A2 — A||_p > 1 
holds. 

The second case is rank($) < s. If ([b]) holds, then it is obvious that ||Ai — A||jr > 1 and 
||A2 — A\\f > 1, so ||$ — Px\\f < and ||$ — Py\\f < 1/^/2 can not hold. 

The third case is rank($) > s. Suppose that the eigenvalues of $ are Ai > A2 > • • • > 
Ajy. If ([b]) holds and ||$ — Px\\f < 1 /a/ 2, then we must have Ai > • • ■ > X s > 1/2 > 
A s+ i > • • • > A at, and eigenvalues Ai, • ■ ■ , X s correspond to the eigenspace span(X). Since 
span (A) /span(y), 

iia 2 - a||f > VWW 2 + a - 1/2) 2 = 1/172, 

so ||$ — Pv||f < l/\/2 can not hold. 
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These three cases show that || T — Px\\f < l/\/2, || T — Py\\f < l/\/2, and © can not hold 
simultaneously. From the discussion in the third case, it is obvious that ||T — -Px||f = l/\/2, 

|(T — Py ||f = 1 /\/2, and ([b]) can hold simultaneously by choosing Ai = ■ • • = A s _i = 1, A s = 
As+i = 1/2, A s+ 2 = • • • = Ajv = 0. 

Consequently, for any Py ■ Py £ Vn,s, T £ R NxN , if ||T — Px\\f < l/v^ and ||T — 
Py Hi? < 1 / y/2, their normal spaces can not intersect at point <f>. If ||T — Px\\f = l/\/2 and 
||T — Py ||f = l/\/2, then there exists a T at which their normal spaces intersect. Thus, for 
Vn,si t = 1/V2, and the condition number 1/r is \/2. m 

The condition number 1/r of Vn,s provides the regularity of this manifold, so its RIP 
is able to be derived. 

4 Proof of Main Result 

PROOF Basically, Theorem [l] is proved by applying the condition number of the manifold 
of the projection matrix Vn,s , calculating the covering number of the set of chords of Vn,s, 
and utilizing the Johnson-Lindenstrauss lemma. 

The set of chords of a manifold Xi is denoted as 

C(M) 1 = { \\X-Y\\ f : X ’ YeM ’ X ^ Y }- ( 7 ) 

From lemma C.l in [13j, we know that for any 0 < T < 3t/ 4, the set C(Bp) is a (4-^/T/r, || • 
|| 2 )-cover of C(Vn,s), where By is defined as 

Bt= U {P + TpV n , s (T)}, (8) 

PeM(V N , s ,T) 

in which AT(Vn,s,T) is the (T,d g )-cover of Vn,s , and TpVn,s(T) := {£ € TpVn,s '■ ||£||f < 
T}. It is easily shown in Part B of the proof of Theorem III. 1 in [13] that the e-cover of 
C(Bp ) satisfies 

\N(C(B T ),e)\ < \M{Vn, 8 ,T)\ (l + 0 +W(V n ^T)\ 2 (l + 7 ) + - 

Theorem 8 in [24] gives that the ( T,d p ) covering number of Grassmann manifold Gr x,s 
is (C 0 /T) s(7V_s) with Co being a universal constant. Remind that the projection distance 
on Grjv, s is defined as 

d 2 p (X,Y ) :=^\\XX T -YY T \\ 2 F . 

Thus, the (\/2 T,d) covering number of Vn,s is also {Cq/T) s ^ n ~ s \ 

For any Z E C(Vx,s), there exist Z EC(P-r) and Z E J\T(C(Bt), e) such that \\{Z—Z)\\% < 
dyjrjr and \\(Z — Z )||2 < e. We then have 

U{Z) II 2 < ||Al(Z -Z)|| 2 + II A{Z - Z)|| 2 + \\A{Z)\\ 2 

<lyfTft + e+\\A(Z)\\ 2 (9) 
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and 


\\A(Z)\\ 2 > \\A(Z)\\ 2 - \\A(Z - Z)J 2 

> \\A(Z)\\ 2 - (\\A(Z - Z)\\ 2 + \\A(Z - Z)h) 
>\\A(Z)\\ 2 -A^T/r-e. (10) 


Equations Q and (10) together with Lemma 1.1 in [T7] (known as the Johnson-Lindenstrauss 


lemma) give that 


and 


sup \\A(Z)|| 2 < sup e + 4 ^/tJt + \\A(Z)\\ 2 
Z£C(Vn,s) ZeAT(C(B T ),e) 


< e + 4 



+ (l + <5) 



( 11 ) 


inf 

ZeC(Vn,s) 


UWh> 


. inf , A(Z) | 2 

Z&JV(C(B T ),e) 


(e + A^T/^) 


>(1-5) 



( 12 ) 


Equations © and ( p~2| ) hold simultaneously with probability exceeding 1 — \J\T{C(Bt), e)| 13 
given that 

4 + 2,0 


m > 


log|JV(C(Br),e)|. 


J 2 /2-S 3 /3 / 

From Lemma [ 2 J we know that r = 1 / a/ 2 - Let 5 = e/2, e = e^/m/(AN), and T = 
rme 2 /(2561V 2 ), then 


mC(B T ),e)\ < 


(V2Co 

\ T 


s(N-s) 


1 + - ) + 


V2C 0 


< 2 


< 2 


I ?*)— 1 L 


N 2 


me- 


2s(N-s) 


2s+l 


(-!) 


2s+l 


N^ 2s+1 


Thus, we conclude that if 
4 + 2/3 


m > 


“ V e 2 /8 — e 3 /24 
2 + /? 
e 2 - e 3 /3 


) (log 2 + 2s(N - s ) log ( 5 i y< ) + (2s + 1) log (l + ) 

O (s{N — s) log ^ 


then VZ E C(Vn,s ), (1 — e)y/rn/N < ||^4(Z)||2 < (1 + e)y/m/N holds with probability 
exceeding 1 — |AA(C(i?r), e)| _/3 . 
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In order to control the probability above, we need a lower bound on \N{C{Bt), e)|. 
Notice that 


\tf{C(B T ),e)\>MV N ,s,T)\[l + - 


> \Af(V N , s ,T)\ = 


N ( Gi n ,s, ~^= 


> 




s(N-s) 


in which the last inequality holds when T < \f2ir/A according to Theorem 8 in[24j. and Co 
is a universal constant. Because T = rme 2 / (256N 2 ) < r/256 = l/(256\/2), we have 

\M(C(B T ),e)\ > (512c 0 ) s(JV_s) = exp(cis(./V - s)). 


Thus, the probability exceeding 

1 — \N(C(Bt), e)| - ^ > 1 — exp(— cij3s(N — s)). 


5 Conclusion 

Subspace projection matrices are low rank matrices with additional structure that allows for 
further reduction of its intrinsic dimension. In this work, the restricted isometry property 
of subspace projection matrix under random orthonormal compression is studied. 

The set of s dimensional subspace projection matrices Vn,s is modeled as an s(N — s ) 
dimensional submanifold in M > NxN , so the main concern is transformed to the problem of 
the stable embedding of Vn,s into M. NxN . One of the key points is the calculation of the 
conditional number 1/r of Vn, s - Once r is obtained, the RIP is able to be established by 
applying covering sets of the set of chords of Vn,s and utilizing the JL lemma. In order to 
calculate the condition number 1 /r, the tangent space and the normal space at every point 
of Vn,s are investigated. The result is that by 0(s(N — s ) log IV) measurements the RIP of 
subspace projection matrix is guaranteed. 

This work is not exhausted. The condition number 1/r of Vn,s provides the regularity 
of this manifold, so its RIP under other random compressions, such as i.i.d. Gaussian com¬ 
pression, could also be established using such condition number. Furthermore, theoretical 
analysis on algorithms for compressed subspace projection matrix recovery could be built 
upon this work. 
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